Self-guided verification of an item

ABSTRACT

A method of providing a level of certification of an attribute of an item is disclosed. A requirement is determined for a level of certification for an attribute of an item. A notification is provided of an evidence item that is to be submitted to evaluate the level of certification of the attribute of the item. The evidence item is received. The level of certification of the attribute of the item is determined based on the received evidence item.

CLAIM OF PRIORITY

This application is a continuation of U.S. application Ser. No.13/838,626, filed Mar. 15, 2013, which is hereby incorporated byreference in its entirety.

TECHNICAL FIELD

The present application relates generally to the technical field of dataprocessing, and, in one specific example, to enabling a seller of anitem to provide proof of an attribute of the item in order to reduce alikelihood that prospective buyers of the item would wish to physicallyinspect the item to verify the attribute of the item.

BACKGROUND

Various network-based publication systems (e.g., EBAY®, AMAZON®, orCRAIGSLIST®) may facilitate the buying or selling of items (e.g., goodsor services) by their users. However, because an item being sold onnetwork-based publication systems is often not available to bephysically inspected (e.g., physically handled or viewed) by aprospective buyer of the item prior to an agreement by the prospectivebuyer to purchase the item, it may be relatively easy for sellers tofalsify information about the items they are selling in comparison to anitem that is being sold by the seller at a brick-and-mortar store.Fraudulent advertising may reduce an amount of trust of buyers andsellers of items on the network-based publication system, which may, inturn, reduce a profit that an owner of the network-based publicationsystem may receive based on completions of transactions on thenetwork-based publication system. Therefore, it may be beneficial to theowner of the network-based publication system to implement systems andmethods to reduce a likelihood that information provided about items bysellers of the items is false information.

BRIEF DESCRIPTION OF THE DRAWINGS

Some embodiments are illustrated by way of example and not limitation inthe figures of the accompanying drawings in which:

FIG. 1 is a network diagram depicting a client-server system withinwhich various example embodiments may be deployed;

FIG. 2 is a block diagram illustrating multiple applications includingthe item-verification applications that, in various example embodiments,are provided as part of the networked system of FIG. 1;

FIG. 3 is a block diagram illustrating example modules of theitem-verification applications of FIG. 2;

FIG. 4 is a flowchart illustrating an example method 400 of providing aprocedure that sellers may use to establish a level of certification foran attribute of an item;

FIG. 5 is a flowchart illustrating an example method 500 of designatingan attribute of an item as having a community-level certification;

FIG. 6 is a flowchart illustrating an example method 600 of designatingan attribute of an item as having a seller-level certification;

FIG. 7 is a flowchart illustrating an example method 700 of designatingan attribute of an item as having an expert-level certification;

FIG. 8 is a flowchart illustrating an example method 800 of sorting aset of references to listings of items based at least in part ondesignations of levels of certifications of attributes of the items; and

FIG. 9 is a block diagram of machine in the example form of a computersystem within which instructions for causing the machine to perform anyone or more of the methodologies discussed herein may be executed.

DETAILED DESCRIPTION

In the following description, for purposes of explanation, numerousspecific details are set forth in order to provide an understanding ofvarious embodiments of the present subject matter. It will be evident,however, to those skilled in the art that various embodiments may bepracticed without these specific details.

Consistent with various embodiments, a method of providing a level ofcertification of an attribute of an item is disclosed. A requirement isdetermined for a level of certification for an attribute of an item. Anotification is provided of an evidence item that is to be submitted toevaluate the level of certification of the attribute of the item. Theevidence item is received. The level of certification of the attributeof the item is determined based on the received evidence item.

This method and various embodiments disclosed herein may be implementedas a computer system having one or more modules (e.g., hardware modulesor software modules). This method and various embodiments disclosedherein may be embodied as instructions stored on a machine-readablemedium that, when executed by a processor, cause the processor toperform the method.

FIG. 1 is a network diagram depicting a system 100 within which variousexample embodiments may be deployed. A networked system 102, in theexample forms of a network-based marketplace or other publicationsystem, provides server-side functionality, via a network 104 (e.g., theInternet or Wide Area Network (WAN)) to one or more clients. FIG. 1illustrates, for example, a web client 106 (e.g., a browser, such as theInternet Explorer browser developed by Microsoft Corporation of Redmond,Wash.) and a programmatic client 108 executing on respective clientmachines 110 and 112. Each of the one or more clients may include asoftware application module (e.g., a plug-in, add-in, or macro that addsa specific service or feature to a larger system. The softwareapplication module may be separate from but tightly-integrated into auser interface and functionality of a software application, such as aspreadsheet application. The software application may be a clientsoftware application executing on a client machine. The softwareapplication module may be optionally deployed in the same environment asthe software application such that the software application module canbe accessed from within the software application. The softwareapplication module may be optionally enabled or disabled within theenvironment (e.g., user interface) of the software application. Thesoftware application module may appear to be a part of the softwareapplication by, for example, providing user interface components orwidgets (e.g., menus, toolbars, menu commands, toolbar commands, and soon) that can be enabled, disabled, added to, or removed from standarduser interface components or widgets provided by the softwareapplication.

An API server 114 and a web server 116 are coupled to, and provideprogrammatic and web interfaces respectively to, one or more applicationservers 118. The application servers 118 host one or more application(s)120. The application servers 118 are, in turn, shown to be coupled toone or more database servers 124 that facilitate access to one or moredatabases 126 or data stores, such as NoSQL or non-relational datastores.

The marketplace applications 120 may provide a number of marketplacefunctions and services to users that access the networked system 102.While the applications 120 are shown in FIG. 1 to form part of thenetworked system 102, in alternative embodiments, the variousapplications 120 may form part of a service that is separate anddistinct from the networked system 102.

Further, while the system 100 shown in FIG. 1 employs a client-serverarchitecture, various embodiments are, of course, not limited to such anarchitecture, and could equally well find application in a distributed,or peer-to-peer, architecture system, for example. The variousapplications 120 could also be implemented as standalone softwareprograms, which do not necessarily have networking capabilities.Additionally, although FIG. 1 depicts machines 130, 110, and 112 asbeing coupled to a single networked system 102, it will be readilyapparent to one skilled in the art that machines 130, 110, and 112, aswell as applications 128, 106, and 108, may be coupled to multiplenetworked systems. For example, the application 128, 106, and 108 may becoupled to applications 120, such as payment applications associatedwith multiple payment processors (e.g., Visa, MasterCard, and AmericanExpress).

The web client 106 accesses the various applications 120 via the webinterface supported by the web server 116. Similarly, the programmaticclient 108 accesses the various services and functions provided by theapplications 120 via the programmatic interface provided by the APIserver 114. The programmatic client 108 may, for example, be a sellerapplication (e.g., the TurboLister application developed by eBay Inc.,of San Jose, Calif.) to enable sellers to author and manage listings onthe networked system 102 in an off-line manner, and to performbatch-mode communications between the programmatic client 108 and thenetworked system 102.

FIG. 1 also illustrates a third-party application 128, executing on athird-party server machine 130, as having programmatic access to thenetworked system 102 via the programmatic interface provided by the APIserver 114. For example, the third-party application 128 may, utilizinginformation retrieved from the networked system 102, support one or morefeatures or functions on a website hosted by the third party. Thethird-party website may, for example, provide one or more promotional,marketplace or payment functions that are supported by the relevantapplications of the networked system 102.

FIG. 2 is a block diagram illustrating multiple applications 120 that,in various example embodiments, are provided as part of the networkedsystem 102. The applications 120 may be hosted on dedicated or sharedserver machines (not shown) that are communicatively coupled to enablecommunications between server machines. The applications 120 themselvesare communicatively coupled (e.g., via appropriate interfaces) to eachother and to various data sources, so as to allow information to bepassed between the applications 120 so as to allow the applications 120to share and access common data. The applications 120 may furthermoreaccess one or more databases 126 via the database servers 124.

The networked system 102 may provide a number of publishing, listing andprice-setting mechanisms whereby a seller may list (or publishinformation concerning) goods or services for sale, a buyer can expressinterest in or indicate a desire to purchase such goods or services, anda price can be set for a transaction pertaining to the goods orservices. To this end, the marketplace applications 120 are shown toinclude at least one publication application 200 and one or more auctionapplications 202 which support auction-format listing and price settingmechanisms (e.g., English, Dutch, Vickrey, Chinese, Double, Reverseauctions etc.). The various auction applications 202 may also provide anumber of features in support of such auction-format listings, such as areserve price feature whereby a seller may specify a reserve price inconnection with a listing and a proxy-bidding feature whereby a biddermay invoke automated proxy bidding.

A number of fixed-price applications 204 support fixed-price listingformats (e.g., the traditional classified advertisement-type listing ora catalogue listing) and buyout-type listings. Specifically, buyout-typelistings (e.g., including the Buy-It-Now (BIN) technology developed byeBay Inc., of San Jose, Calif.) may be offered in conjunction withauction-format listings, and allow a buyer to purchase goods orservices, which are also being offered for sale via an auction, for afixed-price that is typically higher than the starting price of theauction.

Store applications 206 allow a seller to group listings within a“virtual” store, which may be branded and otherwise personalized by andfor the seller. Such a virtual store may also offer promotions,incentives and features that are specific and personalized to a relevantseller.

Reputation applications 208 allow users that transact, utilizing thenetworked system 102, to establish, build and maintain reputations,which may be made available and published to potential trading partners.Consider that where, for example, the networked system 102 supportsperson-to-person trading, users may otherwise have no history or otherreference information whereby the trustworthiness and credibility ofpotential trading partners may be assessed. The reputation applications208 allow a user (for example through feedback provided by othertransaction partners) to establish a reputation within the networkedsystem 102 over time. Other potential trading partners may thenreference such a reputation for the purposes of assessing credibilityand trustworthiness.

Personalization applications 210 allow users of the networked system 102to personalize various aspects of their interactions with the networkedsystem 102. For example a user may, utilizing an appropriatepersonalization application 210, create a personalized reference page atwhich information regarding transactions to which the user is (or hasbeen) a party may be viewed. Further, a personalization application 210may enable a user to personalize listings and other aspects of theirinteractions with the networked system 102 and other parties.

The networked system 102 may support a number of marketplaces that arecustomized, for example, for specific geographic regions. A version ofthe networked system 102 may be customized for the United Kingdom,whereas another version of the networked system 102 may be customizedfor the United States. Each of these versions may operate as anindependent marketplace, or may be customized (or internationalized)presentations of a common underlying marketplace. The networked system102 may accordingly include a number of internationalizationapplications 212 that customize information (and/or the presentation ofinformation) by the networked system 102 according to predeterminedcriteria (e.g., geographic, demographic or marketplace criteria). Forexample, the internationalization applications 212 may be used tosupport the customization of information for a number of regionalwebsites that are operated by the networked system 102 and that areaccessible via respective web servers 116.

Navigation of the networked system 102 may be facilitated by one or morenavigation applications 214. For example, a search application (as anexample of a navigation application) may enable keyword searches oflistings published via the networked system 102. A browse applicationmay allow users to browse various category, catalogue, or inventory datastructures according to which listings may be classified within thenetworked system 102. Various other navigation applications may beprovided to supplement the search and browsing applications.

In order to make listings available via the networked system 102 asvisually informing and attractive as possible, the marketplaceapplications 120 may include one or more imaging applications 216, whichusers may utilize to upload images for inclusion within listings. Animaging application 216 also operates to incorporate images withinviewed listings. The imaging applications 216 may also support one ormore promotional features, such as image galleries that are presented topotential buyers. For example, sellers may pay an additional fee to havean image included within a gallery of images for promoted items.

Listing creation applications 218 allow sellers to conveniently authorlistings pertaining to goods or services that they wish to transact viathe networked system 102, and listing management applications 220 allowsellers to manage such listings. Specifically, where a particular sellerhas authored and/or published a large number of listings, the managementof such listings may present a challenge. The listing managementapplications 220 provide a number of features (e.g., auto-relisting,inventory level monitors, etc.) to assist the seller in managing suchlistings. The listing creation application 218 and listing managementapplications 220 may allow sellers to manage listing in bulk (e.g., in asingle operation, such as by an uploading of a file) and providetemplates for sellers to manage category-specific, vendor-specific, orgeneral-type-specific (e.g., catalog or ticket) listings. One or morepost-listing management applications 222 also assist sellers with anumber of activities that typically occur post-listing. For example,upon completion of an auction facilitated by one or more auctionapplications 202, a seller may wish to leave feedback regarding aparticular buyer. To this end, a post-listing management application 222may provide an interface to one or more reputation applications 208, soas to allow the seller to conveniently provide feedback regardingmultiple buyers to the reputation applications 208.

Dispute resolution applications 224 provide mechanisms whereby disputesarising between transacting parties may be resolved. For example, thedispute resolution applications 224 may provide guided procedureswhereby the parties are guided through a number of operations in anattempt to settle a dispute. In the event that the dispute cannot besettled via the guided procedures, the dispute may be escalated to athird-party mediator or arbitrator.

A number of fraud prevention applications 226 implement fraud detectionand prevention mechanisms to reduce the occurrence of fraud within thenetworked system 102.

Messaging applications 228 are responsible for the generation anddelivery of messages to users of the networked system 102. Thesemessages may, for example, advise users regarding the status of listingsat the networked system 102 (e.g., providing “outbid” notices to biddersduring an auction process or providing promotional and merchandisinginformation to users). Respective messaging applications 228 may utilizeany one of a number of message delivery networks and platforms todeliver messages to users. For example, messaging applications 228 maydeliver electronic mail (e-mail), instant message (IM), Short MessageService (SMS), text, facsimile, or voice (e.g., Voice over IP (VoIP))messages via the wired (e.g., the Internet), Plain Old Telephone Service(POTS), or wireless (e.g., mobile, cellular, WiFi, WiMAX) networks.

Merchandising applications 230 support various merchandising functionsthat are made available to sellers to enable sellers to increase salesvia the networked system 102. The merchandising applications 230 alsooperate the various merchandising features that may be invoked bysellers, and may monitor and track the success of merchandisingstrategies employed by sellers.

The networked system 102 itself, or one or more parties that transactvia the networked system 102, may operate loyalty programs that aresupported by one or more loyalty/promotion applications 232. Forexample, a buyer may earn loyalty or promotions points for eachtransaction established and/or concluded with a particular seller, andmay be offered a reward for which accumulated loyalty points can beredeemed.

Item-verification applications 234, described in more detail below, mayenable sellers of items to back up their assertions pertaining toattributes of the items. For example, a seller may provide proof of anauthenticity of an item or of an operability of an item. In variousembodiments, the proof may come in the form of a result of following aprocedure formulated by experts of a type of the item that the seller isselling to demonstrate that the item has a particular attributeadvertised by the seller of the item.

FIG. 3 is a block diagram illustrating example modules of theitem-verification application(s) 234. An identification module 302 maybe configured to identify a subset of users from a community of usersbased on the subset of users having expertise in an item of a particulartype. A procedure module 304 may be configured to generate one or moreprocedures that a seller of an item or a reviewer of an item shouldcomplete in order to receive a level of certification for an attributeof an item (e.g., a specific instance or physical specimen of a type ofan item that may be listed on a network-based publication system by aseller of the item). A request module 306 may be configured to requestevidence items pertaining to the level of certification of the attributeof the item. An evidence module 308 may be configured to receiverequested items of evidence. A certification module 310 may beconfigured to designate an attribute the item as having a level ofcertification (e.g., based on the receiving of the evidence items). Areception module 312 may be configured to receive a notification thatthe item has been received by a reviewer of the item. A search module314 may be configured to generate references to listing itemscorresponding to keywords in a search query and rank the referencesbased on certification scores corresponding to the references.

FIG. 4 is a flowchart illustrating an example method 400 of providing aprocedure that sellers may use to establish a level of certification foran attribute of an item. Various operations of the method 400 may beperformed by the item-verification application(s) 234. At operation 402,the identification module 302 identifies users that have expertisepertaining to items of a particular type. For example, based on adetermination that the item is an iPhone, the identification module 302may identify one or more users having expertise in smart phones. Theidentification module 302 may identify the users having expertise in thetype of the item based on information provided in profiles of the users,feedback provided by the users, or monitoring of behavior of the users.For example, the identification module 302 may identify that a user hasexpertise in smart phones based on positive feedback received from otherusers on contributions made by the user to an online forum focusing ondiscussions pertaining to smart phones.

At operation 404, the procedure module 304 prompts the identified usersfor information about steps that sellers of items of the particular typeshould complete to receive a level of certification for an attribute ofan item of the particular type. For example, the procedure module 304may prompt the users (e.g., via email, a user interface screen, and soon) for a step that a seller of a particular iPhone should complete tobe eligible to receive a seller-level certification that the iPhone isfunctional or authentic. For example, one of the identified users mayspecify that a seller should upload a video of the particular iPhone tothe system that shows the particular iPhone being turned on and off inorder to receive seller-level certification that the iPhone is operable.Or one of the identified users may specify that the seller should uploadan image of a serial number of the particular iPhone being sold in orderto receive a seller-level certification that the iPhone is authentic.

Or the procedure module 304 may prompt the users for a step that aseller of the particular item should complete to be eligible to receivea community-level certification of the attribute of the particular item.For example, one of the identified users may specify that the usershould receive a positive feedback rating from 95% or higher from theidentified users with respect to an attribute of the item of theparticular type asserted by the seller in order to receive acommunity-level certification of the attribute of the item.

Or the procedure module 304 may prompt the users for a step that aseller of the particular item should complete to be eligible to receivean expert-level certification of the attribute of the particular item.For example, one of the identified users may specify that the usershould post an image of a certificate of authenticity from a third-partyexpert who has personally inspected the item in order to receive anexpert-level certification of the authenticity of the item.

At operation 406, the procedure module 304 may aggregate the informationabout the steps into a master certification procedure for the level ofcertification for the attribute of the item. For example, the proceduremodule 304 may determine, based on feedback received from the identifiedusers, which of the submitted steps should be required of a seller inorder for the seller to be provided with the level of certification forthe attribute of the item. Thus, the master certification procedure mayrepresent a consensus of the identified expert users as to which stepsthe seller should be required to complete in order to receive aparticular level of certification of an attribute of an item of aparticular type.

At operation 408, the procedure module 304 may provide the mastercertification procedure to the sellers. For example, when a seller postsa listing for an item of a particular type, the procedure module 304 mayprovide the seller with steps that the seller may perform to receive alevel of certification for an attribute of the item, such as aseller-level certification, community-level certification, orexpert-level certification of the authenticity of the item.

FIG. 5 is a flowchart illustrating an example method 500 of designatingan attribute of an item as having a community-level certification.Various operations of the method 500 may be performed by theitem-verification application(s) 234. At operation 502, the requestmodule 306 receives a request from a seller that an attribute (e.g., anauthenticity or an operability) of an item be designated as having acommunity-level certification.

At operation 504, the evidence module 308 prompts the seller to provideitems of evidence pertaining to the attribute of the item. In variousembodiments, the items of evidence requested by the evidence module 308are determined by a community of users having expertise pertaining tothe item. For example, if the user is selling a particular watch of aparticular type, an item of evidence that the community of users maydetermine should be submitted by the seller may be a video showing theitem working properly (e.g., ticking).

At operation 506, the evidence module 308 receives the items of evidencefrom the user and provides the items of evidence to the community forreview. For example, the evidence module 308 receives the video from theseller and posts it on a web page corresponding to a listing of the itemon the network-based publication system.

At operation 508, the certification module 310 designates the attributeof the item as having the community-level certification based on apassing of the review by the community. For example, the certificationmodule 310 may receive feedback from the community members on thesufficiency of the evidence provided by the seller to demonstrate theattribute of the item. In various embodiments, the evidence will bedeemed sufficient if a percentage of users of the community who vote onthe sufficiency of the evidence transgresses a threshold. For example,in various embodiments, the evidence may be deemed sufficient by thecertification module 310 if 90% of the users vote that the evidence issufficient for the seller to receive the community-level certification.In this case, the seller may receive a community-level certification ofthe attribute of the item.

FIG. 6 is a flowchart illustrating an example method 600 of designatingan attribute of an item as having a seller-level certification. Variousoperations of the method 600 may be performed by the item-verificationapplication(s) 234. At operation 602, the procedure module 304 providesa seller of the item with a set of steps that the seller may perform tohave an attribute (e.g., an authenticity or an operability) of the itemdesignated as having a seller-level certification. For example, the setof steps may include having the seller submit a photo of a serial numberof the item to demonstrate an authenticity of the item. Or the set ofsteps may include submitting a video that shows the item in a workingcondition.

At operation 604, the evidence module 308 determines that the seller hasperformed the set of steps. For example, the evidence module 308 maydetermine that the seller has submitted a video purporting todemonstrate the item functioning properly. Or the evidence module 308may determine that the seller has submitted a photo purporting to show aserial number of the item.

At operation 606, the certification module 310 designates the attributeof the item as having the seller-level certification (e.g., based on thedetermining that the seller has performed the set of steps). Forexample, the certification module 310 may generate an visualrepresentation of the level of certification that is to be displayed ona web site corresponding to the listing of the item on the network-basedpublication system. In various embodiments, an icon may be associated(e.g., visually) with the attribute of the item to which theseller-level certification applies.

FIG. 7 is a flowchart illustrating an example method 700 of designatingan attribute of the item as having an expert-level certification.Various operations of the method 700 may be performed by theitem-verification application(s) 234. At operation 702, a receptionmodule 312 receives a notification that an item has been received by areviewer. For example, the item may be received by the reviewer from aseller via mail. Or the item may be received by in-person delivery bythe seller to the reviewer. In various embodiments, the reviewer doesnot have expertise in verifying attributes of item of the particulartype.

At operation 704, the procedure module 304 provides the reviewer of theitem with a procedure for determining whether one or more attributes ofthe item may be designated as having an expert-level certification. Invarious embodiments, the procedure is identified by a community of usershaving expertise pertaining to the item. For example, if the item is aniPhone, the procedure may be identified by a community of users havingexpertise in smart phones. Or, if the item is a Toyota RAV4 vehicle, theprocedure may be identified by a community of users having expertise inautomobiles. The procedure may be identified by the community of userssuch that the steps of the procedure may be performed by a reviewer whodoes not have expertise in the type of the item. In other words,although the steps of the procedure may reflect a collective wisdom of acommunity of users who have expertise in the type of the item, the stepsmay be designed or selected by the community of users based on adetermination that by the community of users that a reviewer who is notan expert in the type of the item may be able to perform the steps.

At operation 706, the evidence module 308 receives results pertaining tothe application of the procedure by the reviewer. For example, theevidence module 308 may receive a result from the reviewer that suggeststhat the item is authentic based on an execution of a set of steps toconfirm the authenticity of the item. Or the evidence module 308 mayreceive a result from the reviewer that suggests the item is operablebased on an execution of a set of steps to confirm the operability ofthe item.

At operation 708, the certification module 310 designates the attributeof the item as having an expert-level certification based on thereceiving of the results pertaining to the application of the procedureby the reviewer. For example, the certification module 310 may generatean icon that is to be displayed on a web page corresponding to a list ofthe item on the network-based publication system.

FIG. 8 is a flowchart illustrating an example method 800 of sorting aset of references to listings of items based at least in part ondesignations of levels of certifications of attributes of the items.Various operations of the method 800 may be performed by theitem-verification application(s) 234. At operation 802, the searchmodule receives a search query from a user. For example, the searchmodule may receive one or more keywords entered by the user.

At operation 804, the search module 314 determines a set of referencesto listings of items pertaining to the search query. For example, if thekeywords specified by the user in the search query include “iPhone,” thesearch module 314 may generate references to one or more iPhonescurrently listed on the network-based publication system.

At operation 806, the search module 314 sorts the references based atleast in part on designations of levels of certifications of attributesof the items. For example, the search module 314 may rank a firstreference higher than a second reference based on the first referencepointing to a listing of an item that has a higher certification levelthan the item referred to by the second reference. For example, thesearch module 314 may rank the first reference higher than the secondreference based on the item pointed to by the first listing having anexpert-level certification of the first item's authenticity and the itempointed to by the second listing having only a community-level orseller-level certification of the second item's authenticity.

Or the search module 314 may rank the first reference higher than thesecond reference based on an aggregation of the certified attributes ofthe first item having a higher certification score than the second item.For example, the search module 314 may determine a first certificationscore for the first item of an item based on a determination that thefirst item has a first number of attributes having a first level ofcertification, a second number of attributes having a second level ofcertification, and so on. The search module 314 may also determine asecond certification score for the second item of an item based on adetermination that the second item has a first number of attributeshaving a first level of certification, a second number of attributeshaving a second level of certification, and so on. The scores may bebased on weightings assigned to various attributes based on theirimportance to a buyer (e.g., as measured by a community of users havingexpertise in the type of the item). The search module 314 may then rankthe first reference higher than the second reference based on the firstcertification score being higher than the second certification score.

At operation 808, the search module 314 may transmit the sorted set ofreferences for presentation to the user. In various embodiments, theuser who entered the query pertaining to the references may then bepresented with the sorted list of references.

Certain embodiments are described herein as including logic or a numberof components, modules, or mechanisms. Modules may constitute eithersoftware modules (e.g., code embodied on a machine-readable medium or ina transmission signal) or hardware modules. A hardware module is atangible unit capable of performing certain operations and may beconfigured or arranged in a certain manner. In example embodiments, oneor more computer systems (e.g., a standalone, client or server computersystem) or one or more hardware modules of a computer system (e.g., aprocessor or a group of processors) may be configured by software (e.g.,an application or application portion) as a hardware module thatoperates to perform certain operations as described herein.

In various embodiments, a hardware module may be implementedmechanically or electronically. For example, a hardware module maycomprise dedicated circuitry or logic that is permanently configured(e.g., as a special-purpose processor, such as a field programmable gatearray (FPGA) or an application-specific integrated circuit (ASIC)) toperform certain operations. A hardware module may also compriseprogrammable logic or circuitry (e.g., as encompassed within ageneral-purpose processor or other programmable processor) that istemporarily configured by software to perform certain operations. Itwill be appreciated that the decision to implement a hardware modulemechanically, in dedicated and permanently configured circuitry, or intemporarily configured circuitry (e.g., configured by software) may bedriven by cost and time considerations.

Accordingly, the term “hardware module” should be understood toencompass a tangible entity, be that an entity that is physicallyconstructed, permanently configured (e.g., hardwired) or temporarilyconfigured (e.g., programmed) to operate in a certain manner and/or toperform certain operations described herein. Considering embodiments inwhich hardware modules are temporarily configured (e.g., programmed),each of the hardware modules need not be configured or instantiated atany one instance in time. For example, where the hardware modulescomprise a general-purpose processor configured using software, thegeneral-purpose processor may be configured as respective differenthardware modules at different times. Software may accordingly configurea processor, for example, to constitute a particular hardware module atone instance of time and to constitute a different hardware module at adifferent instance of time.

Hardware modules can provide information to, and receive informationfrom, other hardware modules. Accordingly, the described hardwaremodules may be regarded as being communicatively coupled. Where multipleof such hardware modules exist contemporaneously, communications may beachieved through signal transmission (e.g., over appropriate circuitsand buses) that connect the hardware modules. In embodiments in whichmultiple hardware modules are configured or instantiated at differenttimes, communications between such hardware modules may be achieved, forexample, through the storage and retrieval of information in memorystructures to which the multiple hardware modules have access. Forexample, one hardware module may perform an operation and store theoutput of that operation in a memory device to which it iscommunicatively coupled. A further hardware module may then, at a latertime, access the memory device to retrieve and process the storedoutput. Hardware modules may also initiate communications with input oroutput devices and can operate on a resource (e.g., a collection ofinformation).

The various operations of example methods described herein may beperformed, at least partially, by one or more processors that aretemporarily configured (e.g., by software) or permanently configured toperform the relevant operations. Whether temporarily or permanentlyconfigured, such processors may constitute processor-implemented modulesthat operate to perform one or more operations or functions. The modulesreferred to herein may, in some example embodiments, compriseprocessor-implemented modules.

Similarly, the methods described herein may be at least partiallyprocessor-implemented. For example, at least some of the operations of amethod may be performed by one or more processors orprocessor-implemented modules. The performance of certain of theoperations may be distributed among the one or more processors, not onlyresiding within a single machine, but deployed across a number ofmachines. In some example embodiments, the processor or processors maybe located in a single location (e.g., within a home environment, anoffice environment or as a server farm), while in other embodiments theprocessors may be distributed across a number of locations.

The one or more processors may also operate to support performance ofthe relevant operations in a “cloud computing” environment or as a“software as a service” (SaaS). For example, at least some of theoperations may be performed by a group of computers (as examples ofmachines including processors), these operations being accessible via anetwork (e.g., the network 104 of FIG. 1) and via one or moreappropriate interfaces (e.g., APIs).

Example embodiments may be implemented in digital electronic circuitry,or in computer hardware, firmware, software, or in combinations of them.Example embodiments may be implemented using a computer program product,e.g., a computer program tangibly embodied in an information carrier,e.g., in a machine-readable medium for execution by, or to control theoperation of, data processing apparatus, e.g., a programmable processor,a computer, or multiple computers.

A computer program can be written in any form of programming language,including compiled or interpreted languages, and it can be deployed inany form, including as a stand-alone program or as a module, subroutine,or other unit suitable for use in a computing environment. A computerprogram can be deployed to be executed on one computer or on multiplecomputers at one site or distributed across multiple sites andinterconnected by a communication network.

In example embodiments, operations may be performed by one or moreprogrammable processors executing a computer program to performfunctions by operating on input data and generating output. Methodoperations can also be performed by, and apparatus of exampleembodiments may be implemented as, special purpose logic circuitry(e.g., a FPGA or an ASIC).

The computing system can include clients and servers. A client andserver are generally remote from each other and typically interactthrough a communication network. The relationship of client and serverarises by virtue of computer programs running on the respectivecomputers and having a client-server relationship to each other. Inembodiments deploying a programmable computing system, it will beappreciated that both hardware and software architectures requireconsideration. Specifically, it will be appreciated that the choice ofwhether to implement certain functionality in permanently configuredhardware (e.g., an ASIC), in temporarily configured hardware (e.g., acombination of software and a programmable processor), or a combinationof permanently and temporarily configured hardware may be a designchoice. Below are set out hardware (e.g., machine) and softwarearchitectures that may be deployed, in various example embodiments.

FIG. 9 is a block diagram of machine in the example form of a computersystem 1800 within which instructions for causing the machine to performany one or more of the methodologies discussed herein may be executed.In alternative embodiments, the machine operates as a standalone deviceor may be connected (e.g., networked) to other machines. In a networkeddeployment, the machine may operate in the capacity of a server or aclient machine in server-client network environment, or as a peermachine in a peer-to-peer (or distributed) network environment. Themachine may be a personal computer (PC), a tablet PC, a set-top box(STB), a Personal Digital Assistant (PDA), a cellular telephone, a webappliance, a network router, switch or bridge, or any machine capable ofexecuting instructions (sequential or otherwise) that specify actions tobe taken by that machine. Further, while only a single machine isillustrated, the term “machine” shall also be taken to include anycollection of machines that individually or jointly execute a set (ormultiple sets) of instructions to perform any one or more of themethodologies discussed herein.

The example computer system 1800 includes a processor 1802 (e.g., acentral processing unit (CPU), a graphics processing unit (GPU) orboth), a main memory 1804 and a static memory 1806, which communicatewith each other via a bus 1808. The computer system 1800 may furtherinclude a video display unit 1810 (e.g., a liquid crystal display (LCD)or a cathode ray tube (CRT)). The computer system 1800 also includes analphanumeric input device 1812 (e.g., a keyboard), a user interface (UI)navigation (or cursor control) device 1814 (e.g., a mouse), a storageunit 1816, a signal generation device 1818 (e.g., a speaker) and anetwork interface device 1820.

The storage unit 1816 includes a machine-readable medium 1822 on whichis stored one or more sets of data structures and instructions 1824(e.g., software) embodying or utilized by any one or more of themethodologies or functions described herein. The instructions 1824 mayalso reside, completely or at least partially, within the main memory1804 and/or within the processor 1802 during execution thereof by thecomputer system 1800, the main memory 1804 and the processor 1802 alsoconstituting machine-readable media. The instructions 1824 may alsoreside, completely or at least partially, within the static memory 1806.

While the machine-readable medium 1822 is shown in an example embodimentto be a single medium, the term “machine-readable medium” may include asingle medium or multiple media (e.g., a centralized or distributeddatabase, and/or associated caches and servers) that store the one ormore instructions 1824 or data structures. The term “machine-readablemedium” shall also be taken to include any tangible medium that iscapable of storing, encoding or carrying instructions for execution bythe machine and that cause the machine to perform any one or more of themethodologies of the present embodiments, or that is capable of storing,encoding or carrying data structures utilized by or associated with suchinstructions. The term “machine-readable medium” shall accordingly betaken to include, but not be limited to, solid-state memories, andoptical and magnetic media. Specific examples of machine-readable mediainclude non-volatile memory, including by way of example semiconductormemory devices, e.g., Erasable Programmable Read-Only Memory (EPROM),Electrically Erasable Programmable Read-Only Memory (EEPROM), and flashmemory devices; magnetic disks such as internal hard disks and removabledisks; magneto-optical disks; and compact disc-read-only memory (CD-ROM)and digital versatile disc (or digital video disc) read-only memory(DVD-ROM) disks.

The instructions 1824 may further be transmitted or received over acommunications network 1826 using a transmission medium. Theinstructions 1824 may be transmitted using the network interface device1820 and any one of a number of well-known transfer protocols (e.g.,HTTP). Examples of communication networks include a LAN, a WAN, theInternet, mobile telephone networks, POTS networks, and wireless datanetworks (e.g., WiFi and WiMax networks). The term “transmission medium”shall be taken to include any intangible medium capable of storing,encoding or carrying instructions for execution by the machine, andincludes digital or analog communications signals or other intangiblemedia to facilitate communication of such software. The network 1826 maybe one of the networks 104.

Although an embodiment has been described with reference to specificexample embodiments, it will be evident that various modifications andchanges may be made to these embodiments without departing from thebroader spirit and scope of the present disclosure. Accordingly, thespecification and drawings are to be regarded in an illustrative ratherthan a restrictive sense. The accompanying drawings that form a parthereof, show by way of illustration, and not of limitation, specificembodiments in which the subject matter may be practiced. Theembodiments illustrated are described in sufficient detail to enablethose skilled in the art to practice the teachings disclosed herein.Other embodiments may be utilized and derived therefrom, such thatstructural and logical substitutions and changes may be made withoutdeparting from the scope of this disclosure. This Detailed Description,therefore, is not to be taken in a limiting sense, and the scope ofvarious embodiments is defined only by the appended claims, along withthe full range of equivalents to which such claims are entitled.

Such embodiments of the inventive subject matter may be referred toherein, individually and/or collectively, by the term “invention” merelyfor convenience and without intending to voluntarily limit the scope ofthis application to any single invention or inventive concept if morethan one is in fact disclosed. Thus, although specific embodiments havebeen illustrated and described herein, it should be appreciated that anyarrangement calculated to achieve the same purpose may be substitutedfor the specific embodiments shown. This disclosure is intended to coverany and all adaptations or variations of various embodiments.Combinations of the above embodiments, and other embodiments notspecifically described herein, will be apparent to those of skill in theart upon reviewing the above description.

What is claimed is:
 1. A system comprising: one or more computerprocessors; one or more computer memories; one or more modulesincorporated into the one or more computer memories via acomputer-implemented deployment process, the one or more modulesconfiguring the one or more computer processors of a network-basedpublication system to perform operations for automatically enhancing auser interface for displaying a listing of an item posted on thenetwork-based publication system with a visual representation of a levelof a certification of the item, the operations including: providing anotification of an evidence item that is to be electronically submittedto evaluate the level of certification for the item, the evidence itembeing a data item that is to be stored in a database of thenetwork-based publication system; receiving the electronic submission ofthe evidence item; generating a user interface for prompting a subset ofa community to provide opinions with respect to the evidence item;determining that the requirement for the level of certification for theitem is satisfied based on a consensus of the opinions of the subsetwith respect to the data item indicating that the evidence item issufficient for providing the level of certification; and generating thevisual representation of the level of certification of the item forassociating with a presentation of the listing on a client device. 2.The system of claim 1, the operations further comprising selecting thesubset of the community based on an expertise of the subset with respectto a type of the item.
 3. The system of claim 2, wherein the selectingincludes identifying feedback received from other users in the communityon contributions made by the subset to an online forum focusing ondiscussions pertaining to the type of the item.
 4. The system of claim1, wherein the evidence item includes an image of a serial number of theitem.
 5. The system of claim 1, wherein the evidence item includes avideo verifying a working condition of the item.
 6. The system of claim1, wherein the determining that the requirement for the level ofcertification for the item is satisfied is based on a percentage of thesubset deeming the evidence to be sufficient.
 7. The system of claim 1,wherein the visual representation of the level of certification of theitem includes an icon representing the level of the certification.
 8. Amethod comprising: incorporating one or more modules into one or morecomputer memories via a computer-implemented deployment process, the oneor more modules configuring one or more computer processors of anetwork-based publication system to perform operations for automaticallyenhancing a user interface for displaying a listing of an item posted onthe network-based publication system with a visual representation of alevel of a certification of the item, the operations including:providing a notification of an evidence item that is to beelectronically submitted to evaluate the level of certification for theitem, the evidence item being a data item that is to be stored in adatabase of the network-based publication system; receiving theelectronic submission of the evidence item; generating a user interfacefor prompting a subset of a community to provide opinions with respectto the evidence item; determining that the requirement for the level ofcertification for the item is satisfied based on a consensus of theopinions of the subset with respect to the data item indicating that theevidence item is sufficient for providing the level of certification;and generating the visual representation of the level of certificationof the item for associating with a presentation of the listing on aclient device.
 9. The method of claim 8, the operations furthercomprising selecting the subset of the community based on an expertiseof the subset with respect to a type of the item.
 10. The method ofclaim 9, wherein the selecting includes identifying feedback receivedfrom other users in the community on contributions made by the subset toan online forum focusing on discussions pertaining to the type of theitem.
 11. The method of claim 8, wherein the evidence item includes animage of a serial number of the item.
 12. The method of claim 8, whereinthe evidence item includes a video verifying a working condition of theitem.
 13. The method of claim 8, wherein the determining that therequirement for the level of certification for the item is satisfied isbased on a percentage of the subset deeming the evidence to besufficient.
 14. The method of claim 8, wherein the visual representationof the level of certification of the item includes an icon representingthe level of the certification.
 15. A non-transitory machine-readablemedium storing processor-executable instructions that; when executed byone or more computer processors, cause the one or more computerprocessors of a network-based publication system to perform operationsfor automatically enhancing a user interface for displaying a listing ofan item posted on the network-based publication system with a visualrepresentation of a level of a certification of the item, the operationscomprising: providing a notification of an evidence item that is to beelectronically submitted to evaluate the level of certification for theitem, the evidence item being a data item that is to be stored in adatabase of the network-based publication system; receiving theelectronic submission of the evidence item; generating a user interfacefor prompting a subset of a community to provide opinions with respectto the evidence item; determining that the requirement for the level ofcertification for the item is satisfied based on a consensus of theopinions of the subset with respect to the data item indicating that theevidence item is sufficient for providing the level of certification;and generating the visual representation of the level of certificationof the item for associating with a presentation of the listing on aclient device.
 16. The non-transitory machine-readable medium of claim15, the operations further comprising selecting the subset of thecommunity based on an expertise of the subset with respect to a type ofthe item.
 17. The non-transitory machine-readable medium of claim 16,wherein the selecting includes identifying feedback received from otherusers in the community on contributions made by the subset to an onlineforum focusing on discussions pertaining to the type of the item. 18.The non-transitory machine-readable medium of claim 15, wherein theevidence item includes an image of a serial number of the item.
 19. Thenon-transitory machine-readable medium of claim 15, wherein the evidenceitem includes a video verifying a working condition of the item.
 20. Thenon-transitory machine-readable medium of claim 15, wherein thedetermining that the requirement for the level of certification for theitem is satisfied is based on a percentage of the subset deeming theevidence to be sufficient.